Linearly Constrained Nonsmooth Optimization for Training Autoencoders

A regularized minimization model with $l_1$-norm penalty (RP) is introduced for training the autoencoders that belong to a class of two-layer neural networks. We show RP can act as an exact which shares same global minimizers, local and d(irectional)-stationary points original under mild conditions. construct bounded box region contains at least one minimizer RP, propose linearly constrained (LRP) autoencoders. smoothing proximal gradient algorithm designed solve LRP. Convergence generalized d-stationary point LRP delivered. Comprehensive numerical experiments convincingly illustrate efficiency well robustness proposed algorithm.